When a sensor captures a sequence of successive images, for example in the case of a video, it is known to make an estimate of global inter-image motion. This motion estimate seeks to determine the global motion affecting the image sequence between successive image pairs from the sequence and can correspond to the determination of the motion of the viewing axis of the sensor used.
With such an global motion estimate, image stabilization, image noise removal or even super-resolution algorithms in particular can be implemented.
However, this type of processing can be substantially disrupted when the captured scene comprises objects which are moving within the scene during the image sequence in question.
Image processing systems are then based on implementing a dense but no longer global motion estimate, still called “local motion estimate”.
A dense motion estimate consists of estimating the motion of each point P(x, y), or pixel, of the images from the captured sequence. More specifically, a displacement field {right arrow over (d)}(x, y) is defined based on two images and corresponds to the vectors to be applied respectively to each point of the first image in order to be able to obtain the corresponding point from the second image.
Other than the displacement field {right arrow over (d)}(x, y), the velocity field {right arrow over (v)}(x, y) can be interesting. This velocity field is also called optical flow and is formed by the set of velocity vectors at a given moment for each point of the image. This velocity field is a function of the derivative of the displacement field with respect to a fixed reference.
The dense motion estimation methods are thus based on the assumption of luminance conservation for each point of the scene which is expressed by the apparent motion constraint equation:I(x,y,t)=I(x+dx,y+dy,t+dt)  (1)
where the images from the image series are provided with a reference frame (O, x, y) and axes OX and Oy with which to reference the points P(x, y) from said images. I(x,y,t) is a light intensity value (hereafter called “brightness”) at a point P(x,y) from the scene at a moment of capture t of an image called preceding image Ik-1. An image called following image Ik is captured at an instant t+dt, and dt then corresponds to the interval of time between the two images Ik-1, Ik. Finally, (dx, dy)={right arrow over (d)}(x, y) designates the displacement vector of the point P(x,y) between the instants t and t+dt.
In order to determine the velocity field
            v      →        =                  (                              v            x                    ,                      v            y                          )            =              (                                            ⅆ              x                                      ⅆ              t                                ,                                    ⅆ              y                                      ⅆ              t                                      )              ,also called velocity vector field or motion field, it is possible to expand the above equation just to the first-order.
                              I          ⁡                      (                          x              ,              y              ,              t                        )                          =                              I            ⁡                          (                              x                ,                y                ,                t                            )                                +                      [                                                                                ∂                    I                                                        ∂                    x                                                  ⁢                                  ⅆ                  x                                            +                                                                    ∂                    I                                                        ∂                    y                                                  ⁢                                  ⅆ                  y                                            +                                                                    ∂                    I                                                        ∂                    t                                                  ⁢                                  ⅆ                  t                                                      ]                                              (        2        )            
Equation (2) can be written as follows:
                                                                        ∂                I                                            ∂                x                                      ⁢                          ⅆ              x                                +                                                    ∂                I                                            ∂                y                                      ⁢                          ⅆ              y                                +                                                    ∂                I                                            ∂                t                                      ⁢                          ⅆ              t                                      =        0                            (        3        )            
Finally, equation (3) can be written as follows:
                                                                        ∂                I                                            ∂                x                                      ⁢                          v              x                                +                                                    ∂                I                                            ∂                y                                      ⁢                          v              y                                +                                    ∂              I                                      ∂              t                                      =        0                            (        4        )            
In order to estimate the motion vector at a point P(x, y), this equation (3) then needs to be solved. This equation alone is not however sufficient to determine the motion: it is what is called a poorly stated problem.
In order to get around this difficulty, setting some additional constraints on the desired solution, for example assumptions of local smoothness of the velocity field, is appropriate and various methods are known for moving ahead with this estimation, such as, for example the Horn-Schunck method and the Lucas-Kanade method.
These methods however have disadvantages.
The Horn-Schunck method requires an assumption of smoothness of the velocity field which limits its effectiveness in the presence of discontinuities of the velocity field. Additionally, it is solved by gradient descent and is consequently slow.
The Lucas-Kanade method assumes the velocity field is locally constant. It is robust in the presence of noise, but it has the disadvantage of being sparse, meaning that it is not able to estimate the velocity field in the area of points in whose neighborhood the vector spatial-gradient of the brightness is locally uniform.
“Locally uniform” is understood to mean that the vector spatial-gradients of the brightness near different points from the neighborhood under consideration are either substantially parallel with each other, or substantially zero. Substantially zero is understood to mean that a vector spatial-gradient of brightness considered is less than the image noise.
In the following, (brightness) spatial-gradient is understood to mean any one of the scalar components
  (      for    ⁢                  ⁢    example    ⁢                  ⁢                  ∂        I                    ∂        x              ⁢                  ⁢    or    ⁢                  ⁢                  ∂        I                    ∂        y              )of the (brightness) vector spatial-gradient, which is a vector.
Also the expressions “brightness vector spatial-gradient” and “vector spatial-gradient of brightness” will be used interchangeably.
In the following, (brightness) gradient is understood to mean any one of the scalar components
  (            for      ⁢                          ⁢      example      ⁢                          ⁢                        ∂          I                          ∂          x                      ,                            ∂          I                          ∂          y                    ⁢                          ⁢      or      ⁢                          ⁢                        ∂          I                          ∂          t                      )of the (brightness) vector spatial-temporal gradient, which is a vector.
It is possible to combine the Horn-Schunck method and the Lucas-Kanade method, as is for example detailed in the article “Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods” by A. Brunh, J. Weickert and C. Schnörr. This method however also has disadvantages such as the calculation cost related to the iterative resolution method used in this method or the set weighting of a term called data linkage term which does not consider the degree of smoothness of the calculated displacement field.
The purpose of the invention is to improve the situation.